This invention pertains to the field of three-dimensional imaging and analysis. More specifically, the invention pertains to a system and method for epipolar line-to-line adjustment of stereo pairs.
For both real-world and computer-generated imaging applications, there is a growing need for display techniques that enable determination of relative spatial locations between objects in an image. This is particularly helpful for extracting the 3D topographical information from the stereo image pairs. Once the spatial relations are established, a user can move through the display space and manipulate objects easily and accurately.
One method used to determine spatial relations between objects is binocular stereo imaging. Binocular stereo imaging is the determination of the three-dimensional shape of visible surfaces in a static scene by using two or more two-dimensional images taken of the same scene by two cameras or by one camera at two different positions. Every given point A, in the first image, A, has a corresponding point, B, in the second image, which is constrained to lie on a line called the epipolar line of A. As soon as the correspondence between points in the images is determined, it is possible to recover a disparity field by using the displacement of corresponding points along the epipolar lines in the two images. For example, if two cameras are parallel, the disparity is inversely proportional to the distance from the object to the base line of the cameras. Thus, disparity approaches zero as depth approaches infinity. Once the disparity field is generated and the points in the images are matched, the spatial characteristics of the objects in the images can be calculated using Euclidean geometry.
A related problem in the field of stereo imaging is object recognition and localization. Object recognition and localization includes identifying an object or a particular class of objects, such as identifying a chair, and determining the location of the object in order to maneuver or manipulate the object accordingly. One of the first steps in computer object recognition is connecting as much information as possible about the spatial structure of the object from the analysis of the image. The spatial structure of the object is also important for many other applications, such as three-dimensional object modeling, vehicle navigation and geometric inspection.
Unfortunately, it is very difficult to recover three-dimensional information from a set of 2-d images as this information was lost when the two dimensional image was formed.
The present invention uses stereo imaging techniques to provide alternate views of the same object, thereby enabling determination of object characteristics, such as size and distance. The use of stereo imaging, however, does not resolve the problem associated with identifying the same object in different views, which is another problem requiring careful matching of points in the first image with points in the second image. The matching problem is compounded when the objects within the image are in motion.
In each of these cases, the matching problem can be simplified to a one-dimensional problem if the underlying epipolar geometry were known. What is further needed, then, is a system and method for determining the epipolar geometry between two or more images. Finally, a system and method for aligning the images to the same epipolar line is needed to complete the transformation.
The present invention provides a system and method for determining the epipolar geometry between two or more images (110),(120) taken of the same scene. First, points in the images (110),(120) are matched using an enhanced matching method (500). This method (500) provides highly accurate matching results while maintaining maximum I efficiency.
Once the points are matched, the images (110),(120) are adjusted so that the epipolar geometry of both images (110),(120) are aligned. The images (110),(120) may then be combined into a single stereo image. The present invention can then use other stereo imaging methods to provide alternate views of the same object, thereby enabling greater than determination of object characteristics, such as size and distance.